1. Field of the Invention
The present invention relates generally to the field of computer file systems. More specifically, the present invention discloses a three-dimensional file system based on a virtual node (vnode) architecture.
2. Statement of the Problem
In a distributed computing environment, it is often desirable to share files among a number of users. Normally, sharing is accomplished by setting symbolic links (symlinks) to individual files or directories. Unfortunately, most file system layouts are not organized cleanly, and both sharable and un-sharable files are placed in the same directory. The result is to force the system administrator to set up symlinks at the file level. This requires hundreds or even thousands of symlinks in a typical system. Therefore, it is desirable to provide a three-dimensional file system that allows un-sharable (private) files and sharable files to be organized into different hierarchical levels to simplify system administration. This three-dimensional approach also greatly simplifies archival backups, re-mapping of file systems, and accommodation of diverse file storage devices.
Viewpathing is a concept used by many software development tools (e.g., nmake, build). A common problem faced by software development teams is source code control among members of the team. For example, one engineer may need to modify one portion of the source code while another engineer modifies another portion. It is not desirable for the two modifications to interfere with each other until the modifications are finalized and validated. One solution to address this problem is that each engineer makes their own copy of the source code for development. However, this solution often requires a huge amount of disk space. The viewpathing concept addresses this problem by allowing each user to search for files in different directories in the order specified by the VPATH variable in the user's environment. With the help of viewpathing, each user copies a few necessary files into their own working directory for modification. The viewpath is set so that the software development tools will first search the working directory for files and then search the common source directory tree for any remaining files.
A three-dimensional file system is disclosed by D. G. Korn et al., "The 3-D File System", USENIX 1989 Summer Conference Proceedings, pages 147-156, that extends the concept of viewpathing into the UNIX file system for the purpose of version control in development of multi-release software systems. In particular, Korn's file system enables one directory to be stacked or overlaid one on top of another directory. Any files in the lower-level directory, whose names are not in the top-level directory, appear as if they were part of the top-level directory. In other words, the top-level directory effectively becomes a union of itself and any lower-level directories.
The system described by Korn et al. is based on a viewpath table that is exported into the kernel process table. The kernel process keeps this table during the life of the process. The path name lookup operation (i.e., the operation to find a target file given a path name) searches the files according to the path name given. If a file is not found, the system continues the search in the remaining directories specified in the viewpath table. FIG. 7 illustrates an example of the Korn system. If a user attempts to open "usr/bin/x", the path name lookup operation first searches "/usr/bin". Since "usr/bin/x" does not exist, it looks into the viewpath table and finds that the viewpath for "/usr/bin" is "/n1/bin". The lookup operation continues the search in "/n1/bin" where it found "x". If the user wants "/usr/bin/x/c", the lookup operation first finds "x" (using the same sequence of steps described above) and looks into "/n2/x", then "/n3/x" where it finds "b". To access "a" in the "/n4/x/" directory from the "/usr/bin/x" directory, the user must use a special keyword " . . . " (for example, "ls . . . / . . . / . . . /a"). If the user requests creation of a file "/usr/bin/x/d", the operating system must first create a new directory "x" under "/usr/bin", change directory to the new directory "/usr/bin/x", and then create a new file "d" in that directory.
The system proposed by Korn et al. creates many problems. The UNIX kernel needs to keep a viewpath table for each process, and each viewpath table can be very large. The kernel also needs to remember the history of each path. For example, consider what must happen if the user changes directory to "/usr/bin/x" and creates a file "d". The kernel must remember that the history of the path is "/usr/bin/x", not "/n1/bin/x". This is not practical because the historical logs for each directory can be quite large and complex. Returning to this example, the kernel must first change the path back to "/usr/bin", create a directory "x", change to the new directory, and create the file "x". This creates a new problem in that the directory content is not consistent because directory "x" will appear to have changed. The original directory "x" contained file "a". Due to the fact that the new directory "/usr/bin/x" file is now the top level directory in the stack, directory "x" contains only file "d" and file "a" does not appear to exist. The inconsistent layout of the file system can cause application failures and also confuses users. Changing the current directory to " . . . " from a given directory could lead to any of several different directories depending on the selected viewpath. For example, changing directory to " . . . " from "/usr/bin/x" leads to "/usr/bin", while changing directory to " . . . " from "/n1/bin/x" leads to "/n1/bin", even though "/usr/bin/x" and "/n1/bin/x" could appear to the user as the same directory depending on the viewpath. This is clearly unacceptable.
The system proposed by Korn et al. only allows write operations to directories and files in the top level. All files and directories that are not in the top level can only be accessed on a read-only basis. In addition, files cannot be created or removed from lower-level directories. This creates a number of major problems, including inconsistent permission for files. For example, "/n1/bin/x/a" can be deleted, but "/usr/bin/x/a" cannot be deleted, even though both paths lead to the same file. Operations such as "rm*" will delete files in the top-level directory, but not files in the remainder of the viewpath. Similarly, commands such as "cp" (copy a file) will not perform the operation if the source file is the same name as the target file. The Korn system would require modification of these commands so that the operation is performed if the source file is in a viewpath directory and the target file is in the top-level directory. This not only breaks semantics, but also creates confusion for the user as to whether a file has actually been copied.
A virtual node (vnode) file system for UNIX is described by S. R. Kleiman, "Vnodes: An Architecture for Multiple File System Types in Sun UNIX", USENIX 1986 Summer Conference Proceedings, pages 238-247. This system is intended to allow easy implementation of multiple file system types within the UNIX kernel. Local, remote, and even non-UNIX file systems can be "plugged" into the UNIX kernel through a well defined interface, in much the same way that UNIX device drivers are added to the kernel. Traditional UNIX systems allowed only one type of file system. The Kleiman system creates a new "vnode" software layer 81 above each individual file system type 82, 83, and 84, as shown in FIG. 8. The file system dependent/independent split was done just above the UNIX kernel inode layer, since the inode is the main object for file manipulation in the kernel. The file system independent inode was renamed a virtual node, or "vnode." All file manipulation is done with a vnode object.
Similarly, file systems are manipulated through an object called a "vfs", or virtual file system. The vfs is the analog to the mount table entry in conventional UNIX systems. Each mounted vfs is linked to a list of mounted file systems. The first file system on the list is always the root, as shown in FIG. 9. The private data pointer (vfs.sub.-- data) in the vfs points to file system dependent data. In the 4.2BSD file system 83, vfs.sub.-- data point to a mount table entry.
When a mount system call is performed, the vnode for the mount point is looked up and the vfs.sub.-- mount operation for the file system type is called. If this succeeds, the file system is linked into the list of mounted file systems and the "vfs.sub.-- vnodecovered" field is set to point to the vnode for the mount point. This field is null in the root vfs. Once mounted, file systems are named by the path name of their mount points. Special device names are no longer used because remote file systems do not necessarily have a unique local device associated with them.
The public data fields in each vnode either contain data that is manipulated only by the vfs layer or data about the file that does not change over the life of the file, such as the file type (v.sub.-- type). Each vnode contains a reference count (v.sub.-- count) that is maintained by generic macros called to copy or destroy vnode pointers. When the last reference to a vnode is destroyed, the vn.sub.-- inactive operation is called to tell the vnode's file system that there are no more references. The file system may then destroy the vnode or cache it for later use. The v.sub.-- vfsp field in the vnode points to the vfs for the file system to which the vnode belongs. If a vnode is a mount point, the "v.sub.-- vfsmountedhere" field points to the vfs for the file system. The private data pointer (v.sub.-- data) in the vnode points to data that is dependent on the file system. In the 4.2BSD file system, v.sub.-- data points to an in-core inode table entry.
FIG. 9 shows an example of vnode and vfs object interconnection. In FIG. 9, vnode1 is a file or directory in a 4.2BSD type file system. As such, its private data pointer points to an inode in the 4.2BSD file system's inode table. Vnode1 belongs to vfs1, which is the root vfs, since it is the first on the vfs list. Vfs1's private data pointer points to a mount table entry in the 4.2BSD file system's mount table. Vnode2 is a directory in vfs1, which the mount point for vfs2. Vfs2 is an NFS file system containing vnode3.
Path name traversal is done by a look-up routine, which takes a path name and returns a pointer to the vnode which the path represents. If the path name begins with a "/", the path name traversal starts at the root. Otherwise, it starts at the vnode pointed to by the current directory. The look-up operation traverses the path one component at a time using the vn.sub.-- lookup operation. Vn.sub.-- lookup takes a directory vnode and a specified component as arguments and returns a vnode representing that component. If a directory vnode has "v.sub.-- vfsmountedhere" set, then it is a mount point. When a mount point is encountered going down the file system tree, the lookup operation follows the vnode's "v.sub.-- vfsmountedhere" pointer to the mounted file system and calls the vfs.sub.-- root operation to obtain the root vnode for the file system. Path name traversal then continues from this point. If a root vnode is encountered when following changing to a parent directory (i.e., " . . . "), the lookup operation follows the "vfs.sub.-- vnodecovered" pointer in the vnode's associated vfs to obtain the covered node.
The path name traversal scheme implies that files on remote file systems appear as files within the normal UNIX file name space. Remote files are not named by any special constructs that current programs don't understand. The path name traversal process handles all indirection through mount points, as discussed above. This means that in a remote file system implementation, the client maintains its own mount points. If the client mounts another file system on a remote directory, the remote file system will never see references below the new mount point. Also, the remote file system will not see any " . . . " references at the root of the remote file system. For example, if the client has mounted a server's "/usr" on his "/usr" and a local file system on "/usr/local", then the path "/usr/local/bin" will access the local root, the remote "/usr", and the local "/usr/local" without the remote file system having any knowledge of the "/usr/local" mount point. Similarly, the path "/usr/ . . . " will access the local root, the remote "/usr" and the local root, without the remote file system (or the server) seeing the " . . . " out of "/usr".
Kleiman reports that his virtual node system has been in operation since 1984 and has been released as a product by Sun Microsystems. The system has been used to provide interfaces to the 4.2BSD file system 83, the Sun Network File System (NFS) 84, and an MS-DOS floppy disk file system 82.
3. Solution to the Problem
The present system provides a three-dimensional file system based on a virtual node architecture. Although the system disclosed by Korn et al provides a three-dimensional file system and Kleiman discusses a virtual node file system, none of the prior art references teach or suggest this combination to overcome the problems associated with Korn's three-dimensional file system as described above.
In particular, the present system allows sharable and un-sharable files to be separated in different levels of a stack of directories (a Z-stack). Typically, un-sharable files (i.e., private copies) are at the top of the Z-stack and directories containing sharable files are at the lower portion of the Z-stack. The Z-stack can also be used to organize files hierarchically on different storage devices by their frequency of use, so that rarely-used files are stored on slower devices and frequently-used files are stored on faster devices. Alternatively, this system allows files to be periodically copied to slower backup devices or WORM (write once, read many) devices at off-peak times. Finally, the present system can be employed to allow a user to temporarily install software for a trial period with permanently modifying the user's computer system.